Controlling animated character expressions

ABSTRACT

A system includes a computer system capable of representing one or more animated characters. The computer system includes a blendshape manager that combines multiple blendshapes to produce the animated character. The computer system also includes an expression manager to respectively adjust one or more control parameters associated with each of the plurality of blendshapes for adjusting an expression of the animated character. The computer system also includes a corrective element manager that applies one or more corrective elements to the combined blendshapes based upon at least one of the control parameters. The one or more applied corrective elements are adjustable based upon one or more of the control parameters absent the introduction of one or more additional control parameters.

CLAIM OF PRIORITY

This application is a continuation application and claims priority under35 U.S.C. §120 to U.S. patent application Ser. No. 12/388,806 filed onFeb. 19, 2009 (U.S. Pat. No. 8,207,971 to be issued on Jun. 26, 2012),which claims priority under 35 USC §119(e) to U.S. Patent ApplicationSer. No. 61/141,778, filed on Dec. 31, 2008, the entire contents ofwhich are hereby incorporated by reference.

TECHNICAL FIELD

This document relates to controlling and adjusting expressions ofanimated characters.

BACKGROUND

Parametric models may be generated by a computer for producing animatedcharacters with user-adjustable facial expressions. To simulate theappearance of muscle movement in a character's face, one or more typesof models may be implemented. For one model, a character's face may berepresented with a collection of contour lines and vertices. Dependentupon the desired level of resolution, an extremely large number ofvertices (e.g., tens of thousands, millions, etc.) may be included inthe model. By adjusting the position of the vertices, various facialexpressions may be represented on the character face. For some facialexpressions, relatively few vertex position adjustments are needed whilesignificant adjustments may needed to for representing other facialexpressions.

In some implementations, each facial expression is attained from alinear combination of a selected set of facial expressions (referred toas blendshapes). By adjusting one or more parameters associated with thelinear combination, a range of facial expressions can be created whileutilizing relatively small amounts of computational resources. For eachblendshape, a deformable surface that represents the animatedcharacter's face may be divided into distinct shapes withnon-intersecting boundaries. As such, adjacent shapes that can representmuscular movements tend not to interfere since the shapes do notoverlap, however, the range of producible facial expressions that may belimited.

SUMMARY

The systems and techniques described here relate to using correctiveshapes to control facial expressions of animated characters.

In one aspect, a computer-implemented method includes combining two ormore blendshapes of an animated character, in which each blendshape iscapable of being respectively adjusted by a control parameter. Themethod also includes applying one or more corrective elements to thecombined blendshapes based upon at least one the control parameters.

Implementations may include any or all of the following features.Applying one or more corrective elements may include producing acorrective element to adjust the combined blendshapes. For example, theapplied corrective element may adjust the geometry of the animatedcharacter. Applying a corrective element may include adjusting anon-geometrical feature of the animated character. Applying a correctiveelement may also returning the animated character to a predefined facialexpression. Corrective elements may be applied to independent featuresof the combined blendshapes. Multiple corrective elements may be appliedto the combined blendshapes, based upon one or more of the controlparameters of the blendshapes. Multiple corrective elements may alsoreturn the animated character to a predefined facial expression. Thevalues of one control parameter may be representative of a range offacial expressions of the animated character, a range of muscularmovements of the animated character, a range of simulated movements of asurface of the animated character, a range of movements of one or morejoints of the animated character, and the like. Application of thecorrective element to the combined blendshapes may also be based on userinput.

In another aspect, a system includes an expression manager forrespectively adjusting control parameters of blendshapes to adjust anexpression of an animated character. The system also includes ablendshape manager for combining two or more blendshapes of the animatedcharacter. The system further includes a corrective element manager forapplying at least one corrective element to the combined blendshapesbased upon at least one of the control parameters.

In another aspect, a computer program product tangibly embodied in aninformation carrier and comprising instructions that when executed by aprocessor perform a method that include combining two or moreblendshapes of an animated character. At least one control parameter iscapable of adjusting each blendshape. The method also includes applyingone or more corrective elements to the combined blendshapes based uponat least one of the control parameters.

In another aspect, a system includes a computer system capable ofrepresenting one or more animated characters. The computer systemincludes a blendshape manager that combines multiple blendshapes toproduce the animated character. The computer system also includes anexpression manager to respectively adjust one or more control parametersassociated with each of the blendshapes for adjusting an expression ofthe animated character. The computer system also includes a correctiveelement manager that applies one or more corrective elements to thecombined blendshapes based upon at least one of the control parameters.The one or more applied corrective elements is adjustable based upon oneor more of the control parameters absent the introduction of one or moreadditional control parameters.

Details of one or more implementations are set forth in the accompanyingdrawings and the description below. Other features, aspects andadvantages will be apparent from the description and drawings, and fromthe claims.

DESCRIPTION OF DRAWINGS

FIG. 1 illustrates exemplary ranges of animated character expressions.

FIG. 2 illustrates interfering shapes.

FIG. 3 illustrates use of a corrective element.

FIG. 4 is a diagram of a character development system.

FIGS. 5-7 illustrate applying corrective elements.

FIG. 8 illustrates operations of an expression manager, a blendshapemanager and a corrective element manager.

FIG. 9 is a flowchart of operations of an expression manager, ablendshape manager and a corrective element manager.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

Referring to FIG. 1, facial expressions may be represented by modelsproduced from a linear combination of a selected set of facialexpressions (referred to as blendshapes). By adjusting one or moreparameters associated with the linear combination, a range of facialexpressions can be created while utilizing relatively small amounts ofcomputational resources. For each blendshape, a deformable surface thatrepresents the animated character's face may be divided into distinctgeometries with non-intersecting boundaries. For example, twoblendshapes are illustrated that are each capable of providing tworanges of facial expressions for a relatively simplistic animatedcharacter. Individual expressions may be produced based upon one or moreparameters that control the deformable geometries of the blendshapes. Inthis particular example, one blendshape 100 represents a range of facialexpressions that are bounded by a neutral facial expression 102 and anexpression 104 of happiness (e.g., a smiling face). Similarly, a secondblendshape 106 is capable of producing a range of expressions that arealso bounded by a neutral expression 108 (similar to neutral expression102) and an expression 110 representing an emotion of surprise. In thisexample, the blendshapes 100, 106 are respectively bounded by a neutralexpression and an expression of happiness or surprise, however, in somearrangements other emotions (e.g., sadness, anger, etc.) and boundingexpressions may be represented in the blendshapes.

In general, the animated character's face is represented by a collectionof vertices (e.g., a mesh) with adjustable positions. For example, thepositions of the vertices may be adjusted to illustrate facialexpression changes from a neutral position (e.g., facial expression 102)to another facial expression (e.g., facial expression 104). As such, theblendshapes may include information that represents the positionaldifference of the vertices between one facial expression (e.g., aneutral expression) and another expression (e.g., an expression of anextreme emotion such as happiness). By controlling the verticespositions, the blendshape may be adjusted to produce each of thebounding expressions (e.g., neutral expression 102, expression ofhappiness 104) and expressions that represent linear interpolations ofthe bounding expressions. In some arrangements, the effect of ageometrical adjustment on a single vertex is a three-dimensionaldisplacement that is referred to as a delta. As such, a blendshape maybe considered as a collection of three-dimensionally displaced verticesor deltas.

One or more techniques may be implemented for producing distinctexpressions from the ranges of expressions provided by the blendshapes100, 106. For example, a control parameter (e.g., a weight) that rangesin value from 0 to 1 may be associated with each blendshape. A weightvalue of 0 may be assigned to one expression (e.g., the neutralexpression 102) located at one limit of the expression range and aweight value of 1 may be assigned to the other limit of the range (e.g.,the expression of happiness 104). To illustrate the use of such controlparameter values, respective sliders 112, 114, 116 and 118 represent thevalues assigned to each respective expression. For facial expressionsbetween the two boundary expressions, a weight value between 0 and 1(e.g., 0.8) may be assigned and set by a corresponding slider. As such,control parameter values may be considered as driving a blendshape to aparticular expression. For example, by adjusting a slider (e.g., slider114) to a particular weight value, a particular expression (e.g., theexpression of happiness 104) may be “dialed-up” by a user and renderedon the animated character.

While each of the blendshapes 100 and 106 provides a range of facialexpressions, by combining blendshapes, additional expressions may berepresented on an animated face of a character. For example, bycombining the blendshape 100 (that represents a range expressionhappiness) with the blendshape 106 (that represents a range of emotionsof surprise), additional expressions may be represented. Although, bycombining such expressions, the deltas included in one blendshape maynot properly combine with corresponding deltas of the other blendshapes,thereby causing unrealistic facial expressions to be produced.

Referring to FIG. 2, the blendshape 100 (represented by the facialexpression 104) is illustrated as being combined with the blendshape 106(represented by the facial expression 110) to produce a blendshape thatincludes an expression 200. However, by combining the blendshapes,individual geometries included in the two blendshapes may improperlycombine. Facial muscle movements generally correlate across a variety offacial expressions and combining expressions may produce highlyirregular muscle representations and unrealistic facial expressions. Forexample, the facial expression 104 (dialed up with slider 114) isillustrated as being combined with the facial expression 110 (dialed upwith slider 118) to produce the expression 200, which represents amaximum level of the happiness being combined a maximum level ofsurprise (as illustrated with sliders 202 and 204). Due to thecorrelated geometries included in the two facial expressions 104 and110, features of the expression 200 appear improperly proportioned. Forexample, a feature 206 that represents the mouth of the characterappears abnormally large with respect to the size of the character'sface. As such, one or more facial features of the combined blendshapesmay need to be corrected so that a realistic and recognizable expressionis produced. For such corrections, one or more corrective elements maybe applied to the blendshapes for geometrical and non-geometricaladjustments. For example, one or more geometries (referred to ascorrective shapes) may be added to the combined blendshapes to achieve adesired expression. Non-geometrical adjustments, which may be providedby applied corrective elements, may include texture adjustments (e.g.,skin texture adjustments), color adjustments (e.g., adjusting skinpigmentation), or other similar types of adjustments. Various types ofcorrective elements may be provided for geometric and non-geometricadjustments, for example, blendshapes, textures, normal maps,displacement maps, etc. may be implemented.

Referring to FIG. 3, one type of corrective element, a corrective shape300, is applied to the combined blendshape that includes the facialexpression 200 to produce a realistic facial expression 302 thatproperly represents a combined expression of happiness and surprise. Forexample, application of the corrective shape 300 adjusts the expression200 to reshape one or more facial features (e.g., the character'smouth). One or more techniques may used to define and produce thecorrective shape 300. For example, the corrective shape 300 mayrepresent the positional difference of vertices included in theirregular facial expression 200 and vertices of a desired expression. Assuch, by applying the corrective shape 300 to the expression 200,interference of correlated geometrical movement is substantially removedand, for this example, a corrected facial feature 304 is provided.

Typically, one or more corrective shapes is used to address geometryconflicts for a particular feature or a portion of a feature of acharacter's face. For example, corrective shapes may be produced foradjusting the corner of a character's mouth, an eyebrow, or other typeof facial feature. As such, a large number of corrective shapes may beapplied to correct interfering geometries caused by combiningblendshapes. For example, to animate a relatively complex character facein which subtle facial variations are needed for a particularperformance, an extremely large number of corrective shapes may beneeded. Furthermore, applying one corrective shape to correct one facialfeature (e.g., the corner of a character's mouth) may interfere withanother facial feature (e.g., the upper lip of the character) or evenanother corrective shape, thereby causing the need for even morecorrective shapes.

Similar to dialing up expressions included in a blendshape, a controlparameter (e.g., a weight) may be used to adjust the contribution of acorrective shape. As such, as more and more corrective shapes (e.g.,hundreds, thousands) are applied to an animated character, the number ofcontrol parameters proportionally increase. Correspondingly, a largenumber of control parameter adjustments may be needed to adjust eachcorrective shape for each facial expression of a blendshape. Along withpossibly requiring a significant amount of modeler time to create andapply the corrective shapes, considerable amount of animator time may beneeded for adjusting the corrective shapes to produce the desiredcharacter facial expressions for a performance (e.g., a movie,television program, etc.).

Referring to FIG. 4, a character development system 400 includes acomputer system 402 (or other type of computing device) capable ofgenerating animated characters that use a reduced number of controlparameters for adjusting facial expressions of the characters. Alongwith creating and applying corrective elements (e.g., corrective shapes)to reduce geometry interference, the character development system 400also drives control parameters of blendshapes and corrective elements insuch a manner that less controls parameters need to be adjusted by auser.

Along with components (e.g., interface cards, etc.) for receiving userinput (e.g., from a modeler, animator, etc.) and data (e.g., charactermodels) from various sources (e.g., a library of character models, theInternet, other computer systems, etc.), the computer system 402 alsoincludes memory (not shown) and one or more processors (also not shown)to execute processing operations. A storage device 404 (e.g., a harddrive, a CD-ROM, a Redundant Array of Independent Disks (RAID) drive,etc.) is in communication with the computer system 402 and is capable ofstoring data and providing stored data associated with charactergeneration and character performance production. For example, anillustrative set of blendshapes 406 and corrective shapes 408 arerepresented as being stored on the storage device 404 and retrievable bythe computer system 402 for creating characters and animatingperformances of the characters. Additional types of geometries andinformation may also be stored in the storage device 404.

In this arrangement, to incorporate individual blendshapes orcombinations of blendshapes into animated characters, a blendshapemanager 410 is executed by the computer system 402. In somearrangements, in addition to storing and retrieving blendshapes to andfrom the storage device 404, the blendshape manager 410 may executeother operations for character development. For example, a modeler mayuse the blendshape manager 410 for revising previously createdcharacters (e.g., character facial expressions) or create (e.g.,electronically sculpt) new character expressions that may be stored inthe storage device 404. Automated software packages (e.g., drawing andpainting packages, CAD packages, photograph editing packages, etc.) maybe used in concert with the blendshape manager 410 to produce such newand revised facial expressions. The blendshape manager 410 alsoassociates one or more control parameters with each blendshape. Forexample, one or more weights may be assigned to allow dialing up ofdifferent expressions that may be provided by a blendshape.

To reduce geometry interference (e.g., caused by combining two or moreblendshapes), a corrective shape manager 412 is executed by the computersystem 402. Similarly, for non-geometrical adjustments (or geometricaland non-geometrical adjustments), a corrective element manager may beexecuted. Along with generating corrective shapes to counteractconflicting geometries, the corrective shape manager 412 also associatesone or more control parameters with corresponding corrective shapes. Forexample, one or more variable weights may be assigned to a correctiveshape for adjusting its contribution.

For controlling the blendshapes and corrective shapes, and to reduce thenumber of control parameters that need user interaction, an expressionmanager 414 is also executed by the computer system 402. By reducinguser interaction, less modeler time is needed for producing charactermodels and less animator time is needed for adjusting geometries for acharacter performance. In general, the expression manager 414 allows alarge number of geometries to be controlled by a relatively small numberof control parameters. In one arrangement, the expression manager 414provides one or more high level adjustable control parameters that drivethe control parameters of associated blendshapes and corrective shapes.For example, high level adjustable control parameters may be providedfor particular facial features or portions of a character's face. Assuch, a high level control for a character's mouth, eyebrows, forehead,etc. may be provided for user adjustments. Individual muscles and musclegroups may also be assigned one or more high level control parametersfor adjusting muscle geometries to produce a variety of expressions. Insome arrangements, control parameters may be assigned for controllingthe movement of one or multiple surfaces such as character surfaces(e.g., skin, flesh, etc.), articles associated with a character (e.g.,clothing) or other similar movable surfaces. In still otherarrangements, the control parameters may be assigned to controlling themovement of structures associated with the character. For example, themovements of joints (e.g., shoulder, wrist, elbow, knee, etc.) may beassigned to one or more control parameters. Character emotions are stillanother basis for a high level control parameter that may be used todrive individual blendshapes and corrective shapes associated withindividual expressions or combinations of expressions. For example, onehigh level control parameter may be associated with different levels ofthe emotion happiness and used to drive control parameters of variousgeometries to provide a range of expressions of happiness.

Along with applying corrective shapes to the geometry of a characterbeing developed, information associated with the corrective shapes maybe stored for later retrieval and reapplication. For example, a modelermay apply a corrective shape to the nose of a character that has beendialed up (with a high level control) to express an emotion of extremehappiness. Upon being applied, the corrective shape may be stored in thestorage unit 404 along with the blendshapes to which the correctiveshape has been applied. Information is also stored that records thisassociation between the corrective shape and the blendshapes. Forexample, data may be stored in a shape database 416 that represents theapplication of the corrective shape to the blendshapes for providing anappropriate geometry of a character nose for an expression of extremehappiness. As such, the corrective shape may be retrieved and applied tothe blendshapes for each instance of the character being dialed up topresent an emotion of happiness. Thus, application of a corrective shapeis initiated by use of a high level control to dial a character to anexpression. Furthermore, along with initiating the application of acorrective shape, the high level control may trigger adjustments to thecorrective shape. Continuing with the example, the corrective shape maybe retrieved and applied based on the high level control being dialed toa value associated with an emotion of extreme happiness.Correspondingly, as the high level control is used to reduce thehappiness level being expressed, the geometry of the corrective shapemay be adjusted such that the character's nose is adjusted for thisexpression of reduced happiness.

Referring to FIG. 5, information may be also be stored for applying oneor more corrective shapes to a combination of blendshapes. Asillustrated in FIG. 3, the corrective shape 300 is applied to acombination of blendshapes to produce the facial expression 302 toappropriately represent combined emotions of happiness and surprise. Bystoring the corrective shape 300 (e.g., in the storage device 404), theexpression manager 414 can initiate retrieving and applying thecorrective shape to the combined blendshape for each instance ofreproducing the expression. For example, whenever the sliders 202 and204 are adjusted to values of one, the corrective shape is retrieved andapplied to the combination of blendshapes to produce the facialexpression 302.

In a similar manner, additional corrective shapes may be applied to acombination of blendshapes for other expressions (and stored for latterretrieval and application). For example, upon the slider 204 beingadjusted such that the happiness control parameter has a value of 0.5,the combined blendshape provides a facial expression 500. Due to theapplication of the corrective shape 300, in this instance, theexpression 500 may be unrealistic. In this particular example, a facialfeature 502 that represents the mouth of the character is abnormallylarge and elongated. To adjust the size and geometry of the facialfeature 502, another corrective shape 504 is applied to produce a morerealistic facial expression 506. With the introduction of the correctiveshape 504, the corrective shape 300 compensates (e.g., adjusts deltas)to account for the second corrective shape. Such compensations allow thefacial expression 302 to be returned when the happiness controlparameter is dialed up (via the slider 204) to a value of one. Thus,during such control parameter adjustments (for dialing up facialexpressions), the corrective shapes correspondingly adjust. For example,a parameter (e.g., a numerical value) that represents the geometry ofthe second corrective shape 504 may change correspondingly with thecontrol parameters (being adjusted via the sliders 202, 204, etc.).Alternatively, the parameter associated with the second corrective shape504 may be affected by control parameter adjustments. For example, asthe happiness control parameter is dialed down (via the slider 204) fromthe value of 0.5 to zero, the parameter associated with secondcorrective shape 504 may adjust from a value of one to zero. However, asthe control parameter is dial up (via the slider 204) from the value of0.5 to one, the parameter associated with the second corrective shape503 may retain a value of one.

One or more techniques may be implemented to produce the correctiveshape 504, for example, a modeler may electronically sculpt the facialexpression 506 from the abnormal facial expression 500 to produce thecorrective shape 504. The character development system 400 may alsodetect the differences between the facial expressions 500 and 110 andproduce the corrective shape in an automatic manner. For example, afacial expression that defines a boundary of a blendshape may beidentified by the system 400. Upon an expression of the blendshape beingreproduced due to a particular control parameter setting being detected(e.g., the surprise control parameter set to one and the happinesscontrol parameter being set to zero), one or more corrective shapes areapplied.

Along with applying the corrective shape 504, the corrective shapemanager 412 also initiates information being stored (e.g., in thestorage unit 404) that represents the corrective shape and theassociation of the corrective shape with the blendshapes (and possiblyother corrective shapes) being adjusted. For example, informationrepresentative of the control parameter settings may be stored in theshape database 416 such that upon the settings being detected at anothertime (from the sliders 202, 204), the corrective shape 504 is retrievedand applied to the combination of blendshapes. Furthermore, the appliedgeometry of the corrective shape 504 may be adjusted as the controlparameter settings are adjusted. For example, if the happiness controlparameter is slightly increased, the corrective shape 504 is adjustedsuch that a proportional level of happiness appears in the facialexpression 506. As such, one or more corrective shapes may be appliedand adjusted based upon control parameters associated with two emotions(e.g., surprise and happiness).

Referring to FIG. 6, other methodologies and techniques may be appliedto reduce geometry interference along with the amount of userinteraction needed to set control parameters for adjusting characterexpressions. For example, one or more facial expressions may beconstrained to appear substantially constant for each selected instance.As such, upon returning to a facial expression selected to be invariant,face geometries are returned to reproduce the invariant expression. Forexample, as shown in step A, the sliders 202 and 204 may be fully dialed(to values of one) to produce the expression 302 of combined emotions ofhappiness and surprise. As shown in FIG. 5, and shown here in step B,the slider 204 (associated with happiness) is set to a zero value,thereby returning the animated face to the recognizable expression 506(e.g., by applying the corrective shape 504), as shown in step B.

As mentioned, one or more facial features may be adjusted to alter theexpression being represented. For example, features (e.g., the mouth)may be electronically sculpted to redefine the expression of surprise.As illustrated in an expression 600, the shape of a feature 602 isadjusted such that the represented mouth is opened wider for expressingsurprise, as shown in step C.

In this example, the combined expression of happiness and surprise 302(i.e., in which sliders 202 and 204 are set to values of one) isselected as an invariant expression. As such, upon returning the slidersto control parameter values of one, the invariant expression isreproduced. For example, in step D, the slider 204 associated with thehappiness control parameter is dialed back to a value of one to returnto the invariant expression. However, due to the re-sculpting to producethe surprise facial expression 600, additional geometry interferencesmay appear as the control parameter (associated with slider 204) isadjusted back to a value of one. For example, the re-sculpturing mayresult in an expression 604 that is not equivalent to the invariantexpression 302. Due to the geometry interference, an abnormally largeand disproportionate mouth 606 is present in the expression 604.

To return the expression 604 to an expression equivalent to theinvariant expression 302, one more techniques may be implemented. Forexample, once the expression manager 414 detects that the expression 604differs from the invariant expression 302, another corrective shape 608(or multiple corrective shapes) may be applied to the expression 604 toreproduce the invariant expression 302. Once produced by the correctiveshape manager 412, the corrective shape 608 is applied to the expression604 to produce an expression 610 that is substantially equivalent to theexpression 302 (shown in step E). Typically production and applicationof the corrective shape (e.g., corrective shape 604) or correctiveshapes is executed independent of input from a modeler or animator. Assuch, the production and application of the corrective shape 608 iscontrolled by adjusting the expression control parameters associatedwith the sliders 202 and 204 without additional user input and withoutincreasing the amount of user controls needed to adjust the correctiveshapes and the facial expressions. However, in some implementations,additional user input is used for corrective element (e.g., correctiveshape) production and application.

Along with adding one or more corrective shapes to assure returning toan invariant expression, other techniques may be implemented. Forexample, operations may be initiated for modifying and/or deletingexisting corrective shapes individually or in combination. Further,multiple expressions may be selected as invariant expressions. Forexample, the expression 506, which corresponds to the happiness controlparameter being set to zero (via slider 204) and the surprise controlparameter being set to one (via slider 202), may be defined as anotherinvariant expression. As such, upon dialing up the control parameters(e.g., happiness control parameter set to zero and surprise controlparameter set to one) on the sliders 202, 204, previously producedcorrective shapes (e.g., corrective shape 504) or newly createdcorrective shapes may be used for returning to the invariant expression506. An expression that includes expressions of multiple blendshapes mayalso be defined as an invariant expression. For example, dialing a valueof 0.2 on slider 204 (i.e., 20% happiness control parameter) and 0.5 onslider 202 (i.e., 50% surprise control parameter) may be selected as aninvariant expression. As such, upon these values being set with thesliders 202, 204, one or more corrective shapes (if needed) may beapplied to ensure the invariant expression is produced.

Invariant expressions may also be defined from uniquely createdexpressions (e.g., electronically sculpted). For example, a modeler mayproduce one or a series of facial expressions of a character and defineone or more control parameters for adjusting the expressions. Upon thecontrol parameters being adjusted to settings that correspond to one ofthe sculpted expressions, one or more corrective shapes are applied toreturn the character to the invariant expression.

For situations in which multiple invariant expressions are defined, oneor more techniques may be implemented for producing expressions thatreside between invariant expressions. For example, one or moreestimation techniques (e.g., interpolation, a least-squares estimationprocedure) may be implemented for adjusting geometries (e.g., vertexpositions) and corrective shapes for expressions between two invariantexpressions. In some arrangements, selected invariant expressions mayrepresent extreme emotions. For example, one invariant expression mayrepresent a state of extreme happiness of the character and anotherinvariant may be an extreme expression of surprise or other type ofemotion. Upon adjusting control parameters, one or more interpolationtechniques may be used to calculate a corresponding expression that liesbetween the two extreme expressions.

Referring to FIG. 7, features (or portions) of a facial expression maybe selected for independent geometry adjustments. As such, the characterdevelopment system 400 allows a feature geometry (e.g., an eyebrow) tobe adjusted without causing the geometries of other features to change.In this example, a character expression 700 includes a right eyebrow 702(presented in the figure to the view's left) that may be selected by auser (e.g., a modeler). The facial expression 700 also includes a lefteyebrow 704 (presented in the figured to the view's right). Based uponthe user, the geometry and position of the eyebrow 702 may be adjustedfor providing different facial expressions. For example, as representedin step A, both the right eyebrow 702 and the left eyebrow 704 areoriented substantially horizontal and located relatively near therespective left and right eyes. By the user selecting the right eyebrow702, the geometry and position of the eyebrow may be adjusted forcreating various expressions. To control movement, a control parametermay be assigned to the right eyebrow (e.g., a slider) for raising andlowering the eyebrow. However, raising and lowering the eyebrow 702 mayeffect other portions and features of the expression 700. For example,the orientation, geometry and position of the left eyebrow 704 may beeffected by the independent movements of the right eyebrow 702. Toreduce such interference, corrective shapes may be applied to featuresand portions of the character face such as the left eyebrow 704. Assuch, a portion of the character face 700 may be considered invariantand one or more corrective shapes may be implemented by the correctiveshape manager 412 for geometry retention.

In this scenario, the right eyebrow 702 is selected for being movedindependently with a parameter control, however, one or more features orportions of the character may be joined to the selected feature (e.g.,the eyebrow 702). For example, the left eyebrow 704 may be joined to theright eyebrow 702, and together the two features may be assigned to acontrol parameter for adjusting movement of both features.

As shown in step B, by setting the control parameter, the right eyebrow702 is illustrated as being raised while remaining substantiallyhorizontal in orientation. Based upon the movement of the right eyebrow702, the left eyebrow 704 is moved and changes orientation fromhorizontal to slanted. The movement of the left eyebrow 704 is detectedby the expression manager 414 and one or more corrective shapes(represented as a corrective shape 706) are applied (in step C) by thecorrective shape manager 412 for adjusting the left eyebrow 704 back toa horizontal orientation and position as shown in step A (to counteractthe effects of the right eyebrow 704 movement).

Based upon the application of the corrective shape 706, upon moving theright eyebrow 702, portions of the expression 700 may be effected. Forexample, as illustrated in step D, upon returning the right eyebrow 702to the original position (shown in step A), the left eyebrow 704 isagain moved from the horizontal orientation to a slanted orientation.However, by defining the expression 700 shown in step A as invariant,the expression manager 414 detects the movement of eyebrow 704 andreturns the eyebrow 704 to an orientation to reproduce the invariantexpression. To provide this adjustment, another corrective shape 708 isapplied to the character face 700 such that the expression shown in stepE is equivalent to the expression 700 of step A. However, in somesituations, other corrective shape adjustments may be executed, forexample, with or without adding another corrective shape, one or morecorrective shapes may be deleted or adjusted individually or incombination. As such, individual features and portions of a characterface may be moved independently or jointly (e.g., by assigning one ormore control parameters) to reduce the amount of user input needed toexpression adjustments. Furthermore, by defining one or more invariantexpressions or invariant features of a character face, adjustments maybe implemented (e.g., adding one or more corrective shapes) toappropriately return the character to the proper invariant expression.

Referring to FIG. 8, exemplary interactions among the expression manager414, the corrective shape manager 412 and the blendshape manager 410demonstrate the reduced user interaction needed for adjusting facialexpressions of an animated character. In this example, two blendshapesare combined, in which each blendshape represents ranges of differentemotions. One blendshape provides a range of expressions associated withthe emotion of happiness and the other blendshape provides expressionsassociated with the emotion of surprise. To select among theexpressions, sliders 800, 802 are respectively assigned to eachblendshape by the expression manager 414. Similar to the sliders 202 and204 (shown in FIG. 6), a user may adjust the sliders 800, 802individually or in combination for dialing up expressions of interest.As such, a user (e.g., a modeler, an animator, etc.) can selectexpressions without being aware of the creation and adjustments ofcorrective shapes to adjust expressions. Further, by one or moreexpressions being identified as invariant, corrective shapes may becreated and adjusted (without user awareness) such that the invariantexpressions are consistently recreated upon being selected (e.g., bydialing the appropriate slider or sliders). In this arrangement datafrom the shape database 416 identifies one or more invariant expressions(or invariant facial features) to the expression manager 414.

Upon the sliders 800, 802 being set for selecting an expression from therange of emotions, data representative of the control parameters is sentto the blendshape manager 410 for retrieving the appropriate blendshapeor blendshapes and adjusting corresponding geometries to present theemotional expression of interest. In this example, due to the selectedcontrol parameter values (e.g., a values of one for both the happinesscontrol parameter and the surprise control parameter), the blendshapes100 and 106 are retrieved from the storage device 404 and respectivelyadjusted to produce the expressions 104 and 110. Additionally, theblendshape manager 414 combines the expressions 104, 110 to produce theexpression of interest. Similar to the example shown in FIG. 6, thisparticular combination for blendshapes 104, 110 has been defined as aninvariant expression and is thereby actively adjusted to return to theoriginal expression.

Upon detecting that the selected values of the control parameters (asprovided by the sliders 800, 802), the expression manager 414 providesdata to the corrective shape manager 412 to initiate the creation of oneor more appropriate corrective shapes for being applied to thecombination of blendshape expressions 104 and 110. In this particularexample the corrective shape 300 is produced by the corrective shapemanager 412 (e.g., created, retrieved from the storage device 404, etc.)and applied to the combination of blendshape expressions to produce theinvariant expression 302. As such, by operating just two controlparameters (via the sliders 800, 802), blendshape expressions areselected and combined. Additionally, a corrective shape is produced andapplied to the combined expressions to provide an appropriate andrealistic expression.

Referring to FIG. 9, a flowchart 900 represents some of the operationsof the expression manager 414, the blendshape manager 410 and acorrective element manager (such as the corrective shape manager 412).The operations may be executed by a single computer system (e.g.,computer system 402) or multiple computing devices. Along with beingexecuted at a single site (e.g., at one computer system), operationexecution may be distributed among two or more sites.

Operations include receiving 902 one or more values representative ofone or more corresponding control parameters. For example, interfacedevices (e.g., sliders 800, 802) may be used for selecting values thatrepresent facial expressions within a range of expressions representingan emotion. Rather than emotions, the control parameters values mayrepresent the position, orientation and movement of muscles or musclegroups, facial features (e.g., character mouth, nose, etc.) or otherportions of an animated character (e.g., surfaces such as skin, flesh,clothing, etc.). Operations also include receiving 904 blendshapes(e.g., from the storage device 404) and combining 906 the blendshapesbased upon the values of the control parameters. In someimplementations, blendshape retrieval and combining is provided by theblendshape manager 410, however, such operations may be executed by theexpression manager 414 or the corrective shape manager 412 individuallyor in any combination with the blendshape manager 410.

Operations also include determining 908 if the selected expression (asprovided by the control parameter values) is associated with apreviously defined invariant expression such as illustrated withexpression 302 in FIG. 6. If identified as invariant, operations includeadjusting 910 elements such as geometries such as blendshapes and facialfeatures to return the character facial expression to the invariantexpression. Interpolation techniques may also be implemented forgeometry adjustments. For example, based upon the selected controlparameter values, an expression may be interpolated from one or moreinvariant expressions.

Upon adjusting geometries to account for one or more invariantexpressions, or if not needing to account for an invariant expression,operations include determining 912 if application of one or morecorrective elements (e.g., corrective shapes) is needed. For example,based upon the control parameter values, one or more instances ofblendshape interference may occur and call for at least one correctiveshape being applied. Operations also include determining 914 if one ormore corrective elements (e.g., corrective shapes) need to be created toreduce blendshape interference or whether the current corrective elementor elements may be used and accordingly adjusted to reduce interference.If needed, operations include producing 916 one or more correctiveelements (e.g., corrective shapes) to substantially reduce interference.Upon producing additional corrective elements or determining thatadditional corrective elements are not needed, operations includeapplying 918 the corrective elements (e.g., corrective shapes).Generally, the corrective shape manager 412 executes operations forproducing and applying corrective shapes, however, in someimplementations such operations may be executed individually or incombination with the expression manager 414, the blendshape manager 410or other type of process.

To perform the operations described in flow chart 900, the expressionmanager 414, the blendshape manager 410 and the corrective shape manager412, individually or in combination, may perform any of thecomputer-implement methods described previously, according to oneimplementation. For example, a computer system such as computer system402 (shown in FIG. 4) may execute the expression manager 414. Thecomputer system may include a processor (not shown), a memory (notshown), a storage device (e.g., storage device 404), and an input/outputdevice (not shown). Each of the components may be interconnected using asystem bus or other similar structure. The processor is capable ofprocessing instructions for execution within the computer system. In oneimplementation, the processor is a single-threaded processor. In anotherimplementation, the processor is a multi-threaded processor. Theprocessor is capable of processing instructions stored in the memory oron the storage device to display graphical information for a userinterface on the input/output device.

The memory stores information within the computer system. In oneimplementation, the memory is a computer-readable medium. In oneimplementation, the memory is a volatile memory unit. In anotherimplementation, the memory is a non-volatile memory unit.

The storage device is capable of providing mass storage for the computersystem. In one implementation, the storage device is a computer-readablemedium. In various different implementations, the storage device may bea floppy disk device, a hard disk device, an optical disk device, or atape device.

The input/output device provides input/output operations for thecomputer system. In one implementation, the input/output device includesa keyboard and/or pointing device. In another implementation, theinput/output device includes a display unit for displaying graphicaluser interfaces.

The features described can be implemented in digital electroniccircuitry, or in computer hardware, firmware, software, or incombinations of them. The apparatus can be implemented in a computerprogram product tangibly embodied in an information carrier, e.g., in amachine-readable storage device or in a propagated signal, for executionby a programmable processor; and method steps can be performed by aprogrammable processor executing a program of instructions to performfunctions of the described implementations by operating on input dataand generating output. The described features can be implementedadvantageously in one or more computer programs that are executable on aprogrammable system including at least one programmable processorcoupled to receive data and instructions from, and to transmit data andinstructions to, a data storage system, at least one input device, andat least one output device. A computer program is a set of instructionsthat can be used, directly or indirectly, in a computer to perform acertain activity or bring about a certain result. A computer program canbe written in any form of programming language, including compiled orinterpreted languages, and it can be deployed in any form, including asa stand-alone program or as a module, component, subroutine, or otherunit suitable for use in a computing environment.

Suitable processors for the execution of a program of instructionsinclude, by way of example, both general and special purposemicroprocessors, and the sole processor or one of multiple processors ofany kind of computer. Generally, a processor will receive instructionsand data from a read-only memory or a random access memory or both. Theessential elements of a computer are a processor for executinginstructions and one or more memories for storing instructions and data.Generally, a computer will also include, or be operatively coupled tocommunicate with, one or more mass storage devices for storing datafiles; such devices include magnetic disks, such as internal hard disksand removable disks; magneto-optical disks; and optical disks. Storagedevices suitable for tangibly embodying computer program instructionsand data include all forms of non-volatile memory, including by way ofexample semiconductor memory devices, such as EPROM, EEPROM, and flashmemory devices; magnetic disks such as internal hard disks and removabledisks; magneto-optical disks; and CD-ROM and DVD-ROM disks. Theprocessor and the memory can be supplemented by, or incorporated in,ASICs (application-specific integrated circuits).

To provide for interaction with a user, the features can be implementedon a computer having a display device such as a CRT (cathode ray tube)or LCD (liquid crystal display) monitor for displaying information tothe user and a keyboard and a pointing device such as a mouse or atrackball by which the user can provide input to the computer.

The features can be implemented in a computer system that includes aback-end component, such as a data server, or that includes a middlewarecomponent, such as an application server or an Internet server, or thatincludes a front-end component, such as a client computer having agraphical user interface or an Internet browser, or any combination ofthem. The components of the system can be connected by any form ormedium of digital data communication such as a communication network.Examples of communication networks include, e.g., a LAN, a WAN, and thecomputers and networks forming the Internet.

The computer system can include clients and servers. A client and serverare generally remote from each other and typically interact through anetwork, such as the described one. The relationship of client andserver arises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

A number of implementations have been described. Nevertheless, it willbe understood that various modifications may be made without departingfrom the spirit and scope of the following claims.

What is claimed is:
 1. A computer-implemented method comprising:combining a first deformable geometry of a portion of an animatedcharacter and a second deformable geometry of another portion of theanimated character to form a combined geometry of the animatedcharacter, wherein a first control parameter is capable of adjusting thefirst deformable geometry and a second control parameter is capable ofadjusting the second deformable geometry; and applying at least a firstcorrective geometry, different than the first deformable geometry andsecond deformable geometry, to the combined geometry to correct forcorrelated geometries in the first and second deformable geometries. 2.The computer-implemented method of claim 1, wherein applying the firstcorrective geometry includes producing the first corrective geometry toadjust the combined first deformable geometry and second deformablegeometry.
 3. The computer-implemented method of claim 1, whereinapplying the first corrective geometry includes adjusting the geometryof the animated character.
 4. The computer-implemented method of claim 1further comprising adjusting a non-geometrical feature of the animatedcharacter.
 5. The computer-implemented method of claim 1, whereinapplying the first corrective geometry includes returning the animatedcharacter to a predefined facial expression.
 6. The computer-implementedmethod of claim 1, wherein applying the first corrective geometryincludes applying the first corrective shape to a first feature of thecombined first and second deformable geometry independent of a secondfeature of the combined first and second deformable geometry.
 7. Thecomputer-implemented method of claim 1, further comprising: applying atleast a second corrective geometry to the combined first deformablegeometry and second deformable geometry based upon at least one of thefirst control parameter and the second control parameter.
 8. Thecomputer-implemented method of claim 7, wherein applying the at leastsecond corrective geometry includes returning the animated character toa predefined facial expression.
 9. The computer-implemented method ofclaim 1, wherein values of the first control parameter arerepresentative of a range of facial expressions of the animatedcharacter.
 10. The computer-implemented method of claim 1, whereinvalues of the first control parameter are representative of a range ofmuscular movements of the animated character.
 11. Thecomputer-implemented method of claim 1, wherein values of the firstcontrol parameter are representative of a range of simulated movementsof a surface of the animated character.
 12. The computer-implementedmethod of claim 1, wherein values of the first control parameter arerepresentative of a range of movements of at least one joint of theanimated character.
 13. The computer-implemented method of claim 1,wherein applying the first corrective geometry to the combined firstdeformable geometry and second deformable geometry is based upon userinput.
 14. A system comprising: a computing device comprising: a memoryconfigured to store instructions; and a processor to execute theinstructions to perform operations comprising: combining a firstdeformable geometry of a portion of an animated character and a seconddeformable geometry of another portion of the animated character to forma combined geometry of the animated character, wherein a first controlparameter is capable of adjusting the first deformable geometry and asecond control parameter is capable of adjusting the second deformablegeometry; and applying at least a first corrective geometry, differentthan the first deformable geometry and second deformable geometry tocorrect for correlated geometries in the first and second deformablegeometries.
 15. The system of claim 14, wherein the corrective elementmanager is configured to produce at least one corrective geometry toadjust the combined first deformable geometry and second deformablegeometry.
 16. The system of claim 14, wherein the first correctivegeometry is configured to adjust the geometry of the animated character.17. The system of claim 14, wherein the first corrective geometry isconfigured to adjust a non-geometrical feature of the animatedcharacter.
 18. The system of claim 14, wherein the first correctivegeometry returns the animated character to a predefined facialexpression.
 19. The system of claim 14, wherein the corrective elementmanager is configured to apply the first corrective geometry to a firstfeature of the combined first and second deformable geometry independentof a second feature of the combined first and second deformablegeometry.
 20. The system of claim 14, wherein the corrective elementmanager is configured to apply at least a second corrective geometry tothe combined first deformable geometry and second deformable geometrybased upon at least one of the first control parameter and the secondcontrol parameter.
 21. The system of claim 20, wherein application ofthe at least second corrective geometry returns the animated characterto a predefined facial expression.
 22. The system of claim 14, whereinvalues of the first control parameter are representative of a range offacial expressions of the animated character.
 23. The system of claim14, wherein values of the first control parameter are representative ofa range of muscular movements of the animated character.
 24. The systemof claim 14, wherein values of the first control parameter arerepresentative of a range of simulated movements of a surface of theanimated character.
 25. The system of claim 14, wherein values of thefirst control parameter are representative of a range of movements of atleast one joint of the animated character.
 26. The system of claim 14,wherein the corrective element manager is configured to receive userinput for applying the first corrective element.
 27. A computer programproduct embodied in a non-transitory computer-readable medium andcomprising instructions that when executed by a processor performoperations comprising: combining a first deformable geometry of aportion of an animated character and a second deformable geometry ofanother portion of the animated character to form a combined geometry ofthe animated character, wherein a first control parameter is capable ofadjusting the first deformable geometry and a second control parameteris capable of adjusting the second deformable geometry; and applying atleast a first corrective geometry, different than the combined firstdeformable geometry and second deformable geometry, to correct forcorrelated geometries in the first and second deformable geometries. 28.The computer program product of claim 27, wherein applying the firstcorrective geometry includes producing the first corrective geometry toadjust the combined first deformable geometry and second deformablegeometry.
 29. The computer program product of claim 27, wherein applyingthe first corrective geometry includes adjusting the geometry of theanimated character.
 30. The computer program product of claim 27,wherein applying the first corrective geometry includes adjusting anon-geometrical feature of the animated character.
 31. The computerprogram product of claim 27, wherein applying the first correctivegeometry includes returning the animated character to a predefinedfacial expression.
 32. The computer program product of claim 27, whereinapplying the first corrective geometry includes applying the firstcorrective shape to a first feature of the combined first and seconddeformable geometry independent of a second feature of the combinedfirst and second deformable geometry.
 33. The computer program productof claim 27, wherein the method further comprises: applying at least asecond corrective geometry to the combined first deformable geometry andsecond deformable geometry based upon at least one of the first controlparameter and the second control parameter.
 34. The computer programproduct of claim 33, wherein applying the at least second correctivegeometry includes returning the animated character to a predefinedfacial expression.
 35. The computer program product of claim 27, whereinvalues of the first control parameter are representative of a range offacial expressions of the animated character.
 36. The computer programproduct of claim 27, wherein values of the first control parameter arerepresentative of a range of muscular movements of the animatedcharacter.
 37. The computer program product of claim 27, wherein valuesof the first control parameter are representative of a range ofsimulated movements of a surface of the animated character.
 38. Thecomputer program product of claim 27, wherein values of the firstcontrol parameter are representative of a range of movements of at leastone joint of the animated character.
 39. The computer program product ofclaim 27, wherein applying the first corrective geometry to the combinedfirst deformable geometry and second deformable geometry is based uponuser input.